International Skin Imaging Collaboration-Designated Diagnoses (ISIC-DX): Consensus terminology for lesion diagnostic labeling.

BACKGROUND: A common terminology for diagnosis is critically important for clinical communication, education, research and artificial intelligence. Prevailing lexicons are limited in fully representing skin neoplasms. OBJECTIVES: To achieve expert consensus on diagnostic terms for skin neoplasms and their hierarchical mapping. METHODS: Diagnostic terms were extracted from textbooks, publications and extant diagnostic codes. Terms were hierarchically mapped to super-categories (e.g. 'benign') and cellular/tissue-differentiation categories (e.g. 'melanocytic'), and appended with pertinent-modifiers and synonyms. These terms were evaluated using a modified-Delphi consensus approach. Experts from the International-Skin-Imaging-Collaboration (ISIC) were surveyed on agreement with terms and their hierarchical mapping; they could suggest modifying, deleting or adding terms. Consensus threshold was >75% for the initial rounds and >50% for the final round. RESULTS: Eighteen experts completed all Delphi rounds. Of 379 terms, 356 (94%) reached consensus in round one. Eleven of 226 (5%) benign-category terms, 6/140 (4%) malignant-category terms and 6/13 (46%) indeterminate-category terms did not reach initial agreement. Following three rounds, final consensus consisted of 362 terms mapped to 3 super-categories and 41 cellular/tissue-differentiation categories. CONCLUSIONS: We have created, agreed upon, and made public a taxonomy for skin neoplasms and their hierarchical mapping. Further study will be needed to evaluate the utility and completeness of the lexicon.

Consent and Identifiability for Patient Images in Research, Education, and Image-Based Artificial Intelligence.

This survey study reports the perspectives and preferences of US adults regarding use of photographs of their skin in medical research, education, and development of image-based artificial intelligence (AI).

Expert Agreement on the Presence and Spatial Localization of Melanocytic Features in Dermoscopy.

Dermoscopy aids in melanoma detection; however, agreement on dermoscopic features, including those of high clinical relevance, remains poor. In this study, we attempted to evaluate agreement among experts on exemplar images not only for the presence of melanocytic-specific features but also for spatial localization. This was a cross-sectional, multicenter, observational study. Dermoscopy images exhibiting at least 1 of 31 melanocytic-specific features were submitted by 25 world experts as exemplars. Using a web-based platform that allows for image markup of specific contrast-defined regions (superpixels), 20 expert readers annotated 248 dermoscopic images in collections of 62 images. Each collection was reviewed by five independent readers. A total of 4,507 feature observations were performed. Good-to-excellent agreement was found for 14 of 31 features (45.2%), with eight achieving excellent agreement (Gwet's AC >0.75) and seven of them being melanoma-specific features. These features were peppering/granularity (0.91), shiny white streaks (0.89), typical pigment network (0.83), blotch irregular (0.82), negative network (0.81), irregular globules (0.78), dotted vessels (0.77), and blue-whitish veil (0.76). By utilizing an exemplar dataset, a good-to-excellent agreement was found for 14 features that have previously been shown useful in discriminating nevi from melanoma. All images are public (www.isic-archive.com) and can be used for education, scientific communication, and machine learning experiments.

A reinforcement learning model for AI-based decision support in skin cancer.

We investigated whether human preferences hold the potential to improve diagnostic artificial intelligence (AI)-based decision support using skin cancer diagnosis as a use case. We utilized nonuniform rewards and penalties based on expert-generated tables, balancing the benefits and harms of various diagnostic errors, which were applied using reinforcement learning. Compared with supervised learning, the reinforcement learning model improved the sensitivity for melanoma from 61.4% to 79.5% (95% confidence interval (CI): 73.5-85.6%) and for basal cell carcinoma from 79.4% to 87.1% (95% CI: 80.3-93.9%). AI overconfidence was also reduced while simultaneously maintaining accuracy. Reinforcement learning increased the rate of correct diagnoses made by dermatologists by 12.0% (95% CI: 8.8-15.1%) and improved the rate of optimal management decisions from 57.4% to 65.3% (95% CI: 61.7-68.9%). We further demonstrated that the reward-adjusted reinforcement learning model and a threshold-based model outperformed naïve supervised learning in various clinical scenarios. Our findings suggest the potential for incorporating human preferences into image-based diagnostic algorithms.

Prospective validation of dermoscopy-based open-source artificial intelligence for melanoma diagnosis (PROVE-AI study).

The use of artificial intelligence (AI) has the potential to improve the assessment of lesions suspicious of melanoma, but few clinical studies have been conducted. We validated the accuracy of an open-source, non-commercial AI algorithm for melanoma diagnosis and assessed its potential impact on dermatologist decision-making. We conducted a prospective, observational clinical study to assess the diagnostic accuracy of the AI algorithm (ADAE) in predicting melanoma from dermoscopy skin lesion images. The primary aim was to assess the reliability of ADAE's sensitivity at a predefined threshold of 95%. Patients who had consented for a skin biopsy to exclude melanoma were eligible. Dermatologists also estimated the probability of melanoma and indicated management choices before and after real-time exposure to ADAE scores. All lesions underwent biopsy. Four hundred thirty-five participants were enrolled and contributed 603 lesions (95 melanomas). Participants had a mean age of 59 years, 54% were female, and 96% were White individuals. At the predetermined 95% sensitivity threshold, ADAE had a sensitivity of 96.8% (95% CI: 91.1-98.9%) and specificity of 37.4% (95% CI: 33.3-41.7%). The dermatologists' ability to assess melanoma risk significantly improved after ADAE exposure (AUC 0.7798 vs. 0.8161, p = 0.042). Post-ADAE dermatologist decisions also had equivalent or higher net benefit compared to biopsying all lesions. We validated the accuracy of an open-source melanoma AI algorithm and showed its theoretical potential for improving dermatology experts' ability to evaluate lesions suspicious of melanoma. Larger randomized trials are needed to fully evaluate the potential of adopting this AI algorithm into clinical workflows.

Improving Artificial Intelligence-Based Diagnosis on Pediatric Skin Lesions.

Artificial intelligence algorithms to classify melanoma are dependent on their training data, which limits generalizability. The objective of this study was to compare the performance of an artificial intelligence model trained on a standard adult-predominant dermoscopic dataset before and after the addition of additional pediatric training images. The performances were compared using held-out adult and pediatric test sets of images. We trained two models: one (model A) on an adult-predominant dataset (37,662 images from the International Skin Imaging Collaboration) and the other (model A+P) on an additional 1,536 pediatric images. We compared performance between the two models on adult and pediatric held-out test images separately using the area under the receiver operating characteristic curve. We then used Gradient-weighted Class Activation Maps and background skin masking to understand the contributions of the lesion versus background skin to algorithm decision making. Adding images from a pediatric population with different epidemiological and visual patterns to current reference standard datasets improved algorithm performance on pediatric images without diminishing performance on adult images. This suggests a way that dermatologic artificial intelligence models can be made more generalizable. The presence of background skin was important to the pediatric-specific improvement seen between models. Our study highlights the importance of carefully curated and labeled data from diverse inputs to improve the generalizability of AI models for dermatology, in this case applied to dermoscopic images of adult and pediatric lesions to improve melanoma detection.

Agreement Between Experts and an Untrained Crowd for Identifying Dermoscopic Features Using a Gamified App: Reader Feasibility Study.

BACKGROUND: Dermoscopy is commonly used for the evaluation of pigmented lesions, but agreement between experts for identification of dermoscopic structures is known to be relatively poor. Expert labeling of medical data is a bottleneck in the development of machine learning (ML) tools, and crowdsourcing has been demonstrated as a cost- and time-efficient method for the annotation of medical images. OBJECTIVE: The aim of this study is to demonstrate that crowdsourcing can be used to label basic dermoscopic structures from images of pigmented lesions with similar reliability to a group of experts. METHODS: First, we obtained labels of 248 images of melanocytic lesions with 31 dermoscopic "subfeatures" labeled by 20 dermoscopy experts. These were then collapsed into 6 dermoscopic "superfeatures" based on structural similarity, due to low interrater reliability (IRR): dots, globules, lines, network structures, regression structures, and vessels. These images were then used as the gold standard for the crowd study. The commercial platform DiagnosUs was used to obtain annotations from a nonexpert crowd for the presence or absence of the 6 superfeatures in each of the 248 images. We replicated this methodology with a group of 7 dermatologists to allow direct comparison with the nonexpert crowd. The Cohen κ value was used to measure agreement across raters. RESULTS: In total, we obtained 139,731 ratings of the 6 dermoscopic superfeatures from the crowd. There was relatively lower agreement for the identification of dots and globules (the median κ values were 0.526 and 0.395, respectively), whereas network structures and vessels showed the highest agreement (the median κ values were 0.581 and 0.798, respectively). This pattern was also seen among the expert raters, who had median κ values of 0.483 and 0.517 for dots and globules, respectively, and 0.758 and 0.790 for network structures and vessels. The median κ values between nonexperts and thresholded average-expert readers were 0.709 for dots, 0.719 for globules, 0.714 for lines, 0.838 for network structures, 0.818 for regression structures, and 0.728 for vessels. CONCLUSIONS: This study confirmed that IRR for different dermoscopic features varied among a group of experts; a similar pattern was observed in a nonexpert crowd. There was good or excellent agreement for each of the 6 superfeatures between the crowd and the experts, highlighting the similar reliability of the crowd for labeling dermoscopic images. This confirms the feasibility and dependability of using crowdsourcing as a scalable solution to annotate large sets of dermoscopic images, with several potential clinical and educational applications, including the development of novel, explainable ML tools.

Factors associated with suspected nonmelanoma skin cancers, dysplastic nevus, and cutaneous melanoma among first-time SpotMe screening program participants during 2009-2010.

BACKGROUND: There have been no studies of the American Academy of Dermatology's SpotMe skin cancer screening program to collectively analyze and determine the factors associated with suspected basal cell carcinoma (BCC), squamous cell carcinoma (SCC), dysplastic nevus (DN), and cutaneous melanoma (CM) diagnoses. OBJECTIVE: Describe the demographics, risk factors, and access to care profiles associated with suspected diagnoses of BCC, SCC, DN, and CM among first-time SpotMe screenees during 2009-2010. METHODS: We conducted a cross-sectional analysis of data from the SpotMe skin cancer screenings conducted in 2009 and 2010. We performed multivariable logistic regression analysis for each diagnosis, incorporating standard demographic, access to care, and risk factor variables in the models. RESULTS: Men, those without a regular dermatologist, persons reporting recently changing moles, and those with a personal history of melanoma were at increased risk for each of the suspected diagnoses analyzed. Uninsured persons were at increased risk for suspected malignancies (BCC, SCC, and CM). LIMITATIONS: Lack of histologic confirmation for diagnoses and cross-sectional design. CONCLUSION: Among first-time SpotMe participants, suspected diagnoses of BCC, SCC, DN, and CM shared several associated factors, which may be considered when planning outreach and screening for populations at risk for skin cancer.

Indications for Digital Monitoring of Patients With Multiple Nevi: Recommendations from the International Dermoscopy Society.

Introduction: In patients with multiple nevi, sequential imaging using total body skin photography (TBSP) coupled with digital dermoscopy (DD) documentation reduces unnecessary excisions and improves the early detection of melanoma. Correct patient selection is essential for optimizing the efficacy of this diagnostic approach. Objectives: The purpose of the study was to identify, via expert consensus, the best indications for TBSP and DD follow-up. Methods: This study was performed on behalf of the International Dermoscopy Society (IDS). We attained consensus by using an e-Delphi methodology. The panel of participants included international experts in dermoscopy. In each Delphi round, experts were asked to select from a list of indications for TBSP and DD. Results: Expert consensus was attained after 3 rounds of Delphi. Participants considered a total nevus count of 60 or more nevi or the presence of a CDKN2A mutation sufficient to refer the patient for digital monitoring. Patients with more than 40 nevi were only considered an indication in case of personal history of melanoma or red hair and/or a MC1R mutation or history of organ transplantation. Conclusions: Our recommendations support clinicians in choosing appropriate follow-up regimens for patients with multiple nevi and in applying the time-consuming procedure of sequential imaging more efficiently. Further studies and real-life data are needed to confirm the usefulness of this list of indications in clinical practice.

Digital skin imaging applications, part II: a comprehensive survey of post-acquisition image utilization features and technology standards.

BACKGROUND: Despite the increasing ubiquity and accessibility of teledermatology applications, few studies have comprehensively surveyed their features and technical standards. Importantly, features implemented after the point of capture are often intended to augment image utilization, while technical standards affect interoperability with existing healthcare systems. We aim to comprehensively survey image utilization features and technical characteristics found within publicly discoverable digital skin imaging applications. MATERIALS AND METHODS: Applications were identified and categorized as described in Part I. Included applications were then further assessed by three independent reviewers for post-imaging content, tools, and functionality. Publicly available information was used to determine the presence or absence of relevant technology standards and/or data characteristics. RESULTS: A total of 20 post-image acquisition features were identified across three general categories: (1) metadata attachment, (2) functional tools (i.e., those that utilized images or in-app content to perform a user-directed function), and (3) image processing. Over 80% of all applications implemented metadata features, with nearly half having metadata features only. Individual feature occurred and feature richness varied significantly by primary audience (p < 0.0001) and function (p < 0.0001). On average, each application included under three features. Less than half of all applications requested consent for user-uploaded photos and fewer than 10% provided clear data use and privacy policies. CONCLUSION: Post-imaging functionality in skin imaging applications varies significantly by primary audience and intended function, though nearly all applications implemented metadata labeling. Technical standards are often not implemented or reported consistently. Gaps in the provision of clear consent, data privacy, and data use policies should be urgently addressed.

In vivo tumor immune microenvironment phenotypes correlate with inflammation and vasculature to predict immunotherapy response.

Response to immunotherapies can be variable and unpredictable. Pathology-based phenotyping of tumors into 'hot' and 'cold' is static, relying solely on T-cell infiltration in single-time single-site biopsies, resulting in suboptimal treatment response prediction. Dynamic vascular events (tumor angiogenesis, leukocyte trafficking) within tumor immune microenvironment (TiME) also influence anti-tumor immunity and treatment response. Here, we report dynamic cellular-level TiME phenotyping in vivo that combines inflammation profiles with vascular features through non-invasive reflectance confocal microscopic imaging. In skin cancer patients, we demonstrate three main TiME phenotypes that correlate with gene and protein expression, and response to toll-like receptor agonist immune-therapy. Notably, phenotypes with high inflammation associate with immunostimulatory signatures and those with high vasculature with angiogenic and endothelial anergy signatures. Moreover, phenotypes with high inflammation and low vasculature demonstrate the best treatment response. This non-invasive in vivo phenotyping approach integrating dynamic vasculature with inflammation serves as a reliable predictor of response to topical immune-therapy in patients.

Complementary and alternative therapies in skin cancer a literature review of biologically active compounds.

Complementary and alternative medicine or therapies (CAM) are frequently used by skin cancers patients. Patient's self-administration of CAM in melanoma can reach up to 40%-50%. CAMs such as botanical agents, phytochemicals, herbal formulas ("black salve") and cannabinoids, among others, have been described in skin cancer patients. The objective of this review article was to acknowledge the different CAM for skin cancers through the current evidence, focusing on biologically active CAM rather than mind-body approaches. We searched MEDLINE database for articles published through July 2022, regardless of study design. Of all CAMs, phytochemicals have the best in vitro evidence-supporting efficacy against skin cancer including melanoma; however, to date, none have proved efficacy on human patients. Of the phytochemicals, Curcumin is the most widely studied. Several findings support Curcumin efficacy in vitro through various molecular pathways, although most studies are in the preliminary phase. In addition, the use of alternative therapies is not exempt of risks physicians should be aware of their adverse effects, interactions with standard treatments, and possible complications arising from CAM usage. There is emerging evidence for CAM use in skin cancer, but no human clinical trials support the effectiveness of any CAM in the treatment of skin cancer to date. Nevertheless, patients worldwide frequently use CAM, and physicians should educate themselves on currently available CAMs.

The degradation of performance of a state-of-the-art skin image classifier when applied to patient-driven internet search.

Model Dermatology ( https://modelderm.com ; Build2021) is a publicly testable neural network that can classify 184 skin disorders. We aimed to investigate whether our algorithm can classify clinical images of an Internet community along with tertiary care center datasets. Consecutive images from an Internet skin cancer community ('RD' dataset, 1,282 images posted between 25 January 2020 to 30 July 2021; https://reddit.com/r/melanoma ) were analyzed retrospectively, along with hospital datasets (Edinburgh dataset, 1,300 images; SNU dataset, 2,101 images; TeleDerm dataset, 340 consecutive images). The algorithm's performance was equivalent to that of dermatologists in the curated clinical datasets (Edinburgh and SNU datasets). However, its performance deteriorated in the RD and TeleDerm datasets because of insufficient image quality and the presence of out-of-distribution disorders, respectively. For the RD dataset, the algorithm's Top-1/3 accuracy (39.2%/67.2%) and AUC (0.800) were equivalent to that of general physicians (36.8%/52.9%). It was more accurate than that of the laypersons using random Internet searches (19.2%/24.4%). The Top-1/3 accuracy was affected by inadequate image quality (adequate = 43.2%/71.3% versus inadequate = 32.9%/60.8%), whereas participant performance did not deteriorate (adequate = 35.8%/52.7% vs. inadequate = 38.4%/53.3%). In this report, the algorithm performance was significantly affected by the change of the intended settings, which implies that AI algorithms at dermatologist-level, in-distribution setting, may not be able to show the same level of performance in with out-of-distribution settings.

Interest and Utility of MC1R Testing for Melanoma Risk in Dermatology Patients with a History of Nonmelanoma Skin Cancer.

Public access to genetic information is increasing, and community dermatologists may progressively encounter patients interested in genetic testing for melanoma risk. Clarifying potential utility will help plan for this inevitability. We determined interest and uptake of genetic risk feedback based on melanocortin receptor gene (MC1R) variants, immediate (two weeks) responses to risk feedback, and test utility at three months in patients (age ≥ 18, with a history of nonmelanoma skin cancer). Participants (N = 50) completed a baseline survey and were invited to consider MC1R testing via the study website. Testing interest and uptake were assessed through registration of test decision, request of a saliva test kit, and kit return (all yes/no). Immediate responses to risk feedback included feedback-relevant thoughts, emotions, communication, and information seeking after result receipt; test utility outcomes included family and physician communication and information seeking. Results indicated good retention at both time points (76%; 74%). Half (48%) logged onto the study website, and of these, most (92%) chose testing and (95%) returned a saliva sample. After two weeks, most (94%) had read all the risk feedback information and distress was low (M = 8.81, 7-28, SD = 2.23). Many (69%) had talked with their family about the results. By three months, most had spoken with family (92%) and physicians (80%) about skin cancer risk. Physician communication was higher (70%) in those tested versus those not tested (40%, p = 0.02). The substantial interest and promising outcomes associated with MC1R genetic testing in dermatology patients inform intervention strategies to enhance benefits and minimize risks of skin cancer genetic testing.